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ABSTRACT 

INSCAL multidimensional scaling, an objective 
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measure is obtained after student exposure to the material, INSCAL 
provides an indication of the correspondence between experts' and 
students' understanding of concept interrelationships. Major findings 
from the final phase indicated that: (1) alternative sequences of 
instructional material influenced student performance; (2) pictorial 
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reading aptitude was significantly related to student performance 
under both pictorial and verbal print presentation modalities; and 
(5) instruction sequences did not interact with student aptitude. 
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Background 

To be cost-effective, computer-based instruction must maximize 
student attainment of training objectives and offset computer costs 
through minimization of student training time to acliieve each oujec- 
tive, Tlie question of how coiiplex teclonical information is best 
presented, in what form, and in what sequence, impact greatly upon 
the optimization of con^uter efficiencies for large numbers of students. 
Within computer based instructional systems, the effects of alternative 
instructional sequences upon student time to attain instructional objec- 
tives is important, but largely unknown. Traditionally, instructional 
sequences are based upon intuitive versus systematic scientific approaches. 
To expedite student progress, identification and validation of performance 
facilitating and debilitating instructional sequences must be determined 
if training time efficiencies are to be realized. Promising scientific 
methods are now available which allow con^lexity of teclinical information 
to be objectively measured, and subsequently modified to produce infor- 
mation content and sequences effective for students who var>'' in charac- 
teristics, e.g., aptitude level, A multivariate study was thus required 
to determine effects and interralationships among alternative instructio- 
nal sequences, technical information difficulty levels, media presentation 
modalities and student aptitude levels upon multiple measures of perfor- 
mance . 

Objectives were: to investigate the utility of an objective method- 
ology (Inscal) for formulating effective technical information sequences; 
to test effectiveness of alternative instructional sequences in verbal 
and pictorial presentation modes; to determine if the effectiveness of 
instructional sequences varies as a function of the difficulty level of 
technical material; to determine if some instructional sequences are 
more performance- facilitating for low-aptitude students whereas other 
sequences are more effective for high-aptitude students; to determine 
effects of verbal print versus pictorial information sequences upon the 
performance levels of high- and low-aptitude students. 

Approach 

Technical materials (Structure of Matter and Oscilloscope Operations) 
were selected and detemined to constitute significantly different 
difficulty levels. Inscal multidimensional scaling methodology was 
applied to 50 minute course segments by means of concept similarity 
ratings obtained from technical experts within the academic and military ' 
communities. Computer algorithms were developed to provide two different 
instructional sequences for each instructional package. Also, pictorial 
and verbal print material versions of each instructional package were 
generated. Measures of student performance for each instructional 
package included: (1) verbal print multiple choice tests, (2) pictorial 
multiple choice tests developed to correspond to verbal multiple choice 
tests, (3) Concept Cloze, as a measure of student conprehension of 
technical material covered, and (4) Student/Expert Understanding of 
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Concept Interrelationships, as a post-instructional measure of the degree 
to which student understanding of technical concepts corresponded to the 
experts^ understanding of the interrelationships among technical concepts. 
High and low student aptitude classifications were assessed by the Delta 
Reading Vocabulary measure. 

Findings and Conclusions 

Instructional sequence alternatives developed from the Inscal method 
resulted in statistically significant but otherwise small student perfor- 
mance differences ajuong the instructional sequences. Tlius, how technical 
information is sequenced does contribute to subsequent student performance; 
however, the magnitude of these differences under the conditions of the 
present study was not great (less than 51 in student performance) . Though 
media presentation modalities (pictorial versus verbal print) were found 
to be statistically significant, student performance differences were 
found to be marginal. Thus, the findings from the present study indicate 
no practical performance gains attributable to pictorial presentation as 
opposed to verbal print presentation for low-aptitude students as reported 
by other investigators. 

Instructional sequencing -alternatives were not found to interact with 
high or low student aptitude, media mode (pictorial versus print)or diffi- 
culty level of technical material. Hence, instructional sequences similar 
to those employed in the present study may not unduly influence the per- 
formance of students even if those students differ in aptitude. 

Results of a four-way analysis of variance indicated that high scores 
on the Delta Reading Vocabulary aptitude measure performed substantially 
better on all dependent variables to include the pictorial multiple choice 
test than did low-aptitude students. Within the low-aptitude group, no 
evidence was found to support the contention that pictorial multiple 
choice tests result in better performance than verbal tests for low- 
aptitude students. Hence, it is unlikely that pictorial tests will sub- 
stantially improve low-aptitude student performa^ice. 

In all conditions, males outperformed females on all performance 
measures. However, performance differences were primarily attributable 
to the difficulty factor inherent in the oscilloscope as opposed to the 
structure of matter technical material. Thus, if female technical trainees 
are to be increasingly assigned to courses where unfamiliar and difficult 
technical concepts are taught, student aptitude and motivational factors, 
e.g., interest, should be given greater consideration when technical 
materials are developed or revised. 

In conclusion, the Inscal method of developing both improved measures 
of technical information complexity and performance- facilitating instruc- 
tional sequences, also offers several other advantages. This objective 
method can be applied to new technical material at the time experts write 
it. Further, alternative instructional sequences can be tailored to the 
needs of subgroups of students in an individualized manner. Other benefits 
are discussed in the Conclusions section of this report. 
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PREFACE 



The oontributlons of knowledge and tdine irade by numeroxas 
individuals in the local oormwnities, academic and military 
environs irtpelled the development and successful coirpletian 
of the Phase II study. Tlie leadership of Texas Christian 
University's Colonel Bearden (USA PCfTC) and Lt. Colonel Seed 
(USAF POTC) and University of Texas at Arlington's Lt. Colonel 
Wteems (USAF K3TC) in the recruitment of the RDTC st\:ident 
subjects was of vital significance. Coqperation of the Hill 
County Junior College's (Cleburne, Texas) administration in 
enco^jraging student participation was deeply appreciated. In 
addition, the excellent oonbined coordination and cooperation 
of the Texas Christian Psychology Department faculty in 
providing the general psychology student subjects played an 
irrportant role in cotpleting the Phase II experimentation. 

Noteworthy contributions were made by Dr. Gerald Deignan, 
Technical Training Division, Lo^Try AFB, Colorado by his 
constant involvement, deep concern and ready assistance in 
all areas of the research. 
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PtIASE II: THE MAIN EXPERINENT 



One of the major concliisions dra\m from an analysis of 
previous investigations into the effects of instructional 
sequencing upon student performance is the inconsistency 
of the findings (AFHRL-^R-73-51 (I), Factors Related to 
Developing Optimal Instructional Sequences: Phase I) . To 
summarize briefly, sane studies have found that perfoiniianoe 
is apparently influenced by the particular sequence in 
which instructional material is presented, while others 
have not. The variety of procedures, stimuli, and response 
measures used in prior e:<periments has made it difficult 
to determine the conditions under vdiich instructional 
sequencing is an irrportant variable. However, two 
critical factors influencing the corf licting outcorres of 
these e5<periments have been identified: 

(1) lack of systematic procedures for developing 
informational sequences, aiid 

(2) minimal attention to individual differences in 
information processing styles and capacity. 

The primary purpose of the present e^qperiinent was to 
incorporate solutions to these two difficulties in an 
assessment of the effects of instructional sequencing upon 
performance. Additionally, two variables thought to inter- 
act with the effects of sequence and with individual 
differences have been included in an attenpt to delimit 
conditions which attenuate or enhance sequence effects. 
These two variables, difficult^ of the instructional 
material and mode of presentation (verbal versus pictorial/ 
graphic) , are also of general iitportance in the design of 
instructional procedures. 

Specifically, the itiain e5<periment has been designed: 

(1) to investigate the utility of multidimensional 
scaling as a method for describing the information 
structure of Air Force instructional material and to 



investigate its potential as a basis for forming effective 
instructional sequences. As a result of the pilot studies 
discussed in AFHRI>TR-73-51 (I) , the previously cited Phase 
I report, two instructional sequences, algorithmically 
formed from multidimensional scaling solutions of similarity 
judgments by "e^^^erts" in the content areas, were selected 
for assessment in the main e>55erin[>ent along v/ith the 
original Air Force sequences. 

(2) to determine if interaction effects exist between 
information sequence requirements and a measure of intel- 
lectual ability. The principal measure of ability eirployed 
is the Delta Heading Vocab;alary Test. 

(3) to determine the effect of two modes of presenta- 
tion, verbal and pictorial/graphic, on test performance, 
especially in interaction with informational sequences and 
individual differences. 

(4) to assess the interaction of instructional 
material difficulty with other independent variables. Two 
Air Force instructional packages varying in difficulty 
have been selected for inclusion on the basis of pilot 
work reported in AFHRL-TR-73-51 (I) . 

(5) to investigate the utility of two new performance 
measures in the assessment of the inpact of instructional 
manipulations. A modified version of the "cloze" procedure 
and a similarity judgment methodology were developed for 
\jse mth the Air Force material. Discussion of these 
dependent measures, along v/ith standard multiple choioe 
tests, shall be elaborated upon in the subsequent section. 

Due to the conflicting nature of the results of 
previous experijnents on sequencing and due to the explora- 
tory nature of this stu*^, no specific hypotheses have 
been formulated. 

Method 

This section will be divided into siibsections dealing 
with: experimental desi.gn, subjects, stimailus materials, 
dependent measures, individijal difference measiices, and 
procedure. 
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Experimantal Design . A fully crossed, three factor, 
between-subjects design v/as enployed in this experiment 
(see Figiire 1) . The three factors were: instructional 
sequence (three levels designated original, proximal, and 
non-proxirral) , node of presentation (two levels designated 
verbal and pictorial/graphic) , and instructional package 
difficulty (two levels designated Oscilloscope Operation 
and Characteristics of Matter) . * Fifteen subjects were 
included in each of the twelve cells of the design. This 
nurtiber of subjects was considered sufficient to allav for 
the investigation of the effects of individual differences 
in intellectual ability. 



r 

Mode of 
.Presentation 


Package 
Difficulty 


Instructional Sequence 


Verbal 


Oscilloscope 


Original 
Air Force 


Proximal 


Non- 
Proximal 


15 Ss 


15 Ss 


15 Ss 


Characteristics 
of Matter 


15 Ss 


15 Ss 


15 Ss 


Pictorial/ 
Graphic 


Oscilloscope 


15 Ss 


15 Ss 


^ 15 sJs 


Characteristics 
of Matter 


15 Ss 


15 Ss 


15 Ss 



FIGURE 1 



Experimental Design 

Siipjects . Qn^ hundred and eighty college- students 
served as subjects in this es^jeriment. Due to lack of 
a sufficiently large hoannogeneous body of available subjects, 
it was necessary to recruit individuals fraxi a variety of 
sources. The subject mix can be characterized as follows: 
50 males from the Air Foroe and Arny RDTC \anits at Texas 
Christian University and the University of. Texas at 
Arlington, 72 males from general psychology classes at 
Texas Christian University and Hill Covmty Junior College, 
and 58 females from general psychology classes at Texas 
Christian University. General Psychology students v?ere given 
e^^^eriiiental credit as partial fulfillment of their course 
requirements, while ROTC students x^jere paid at a rate of $2.00 
per hoiar. These s\±)jects v;ere distributed as systotatically 
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as possible acaross the eJ^^erimental cx;)ncaitions in order to 
avoi^d biasing the treatment effects and in order to set up 
a basis for exploring the effects of ilndividual differ- 
ences on the dependent ireasures. Since the military ittay 
experience an influx of females in the future, male-female 
differences in this experiment will be of particular 
interest. In each cell of this experiment there were four 
to six females and nine to eleven males. 

Stimulu s Jyiaterials . Based on the pilot work discussed 
in AFHEL-TR-73-51 (I) , tavo sets of Air Force instructional 
material, varying in difficulty of aoqui,sition, were 
selected for inclxision in this experiment. These "b^ sets 
were: 

(1) Characteristics of Matter, {]?ages 1-22 of the 
document labeled Precisian Measxaring Equipment Specialist, 
D C Circuit Analyses Block II) , 

(2) Oscilloscope Operation > (pages 1-18 of the 
Maintenance Electronics Operation) . Thie resxolts of Pilot 
Study 2 indicated that the Oscillosoope Operation package 
led to significantly poorer performance on the dependent 
measures than did the Characteristics of Matter package. 

The original Air Force instructioncil packages were 
in primarily verbal form. T'o investigate the effects of 
pictorial/graphic as well as; verbal presentation, these 
packages were converted intc:) primarily pictorial represen- 
tation. Approximately seven-teen key conoepte wei:e 
derived frcm each of the packages, and a oonoept by 
concept matching of the information was maintained between 
the verbal and pictorial/graphic modes. These packages 
were prepared in booklet form containing approximately one 
concept per page. Exaitples of these stimulus packages were 
presented in Annex A of the AEHFa>TR-73-51 (I) . 

Three instructional sequences were xised in the present 
e^q^erimsnt: (1) proximal (2) non-proximal and (3) the 
original Air Force sequence. Sequence ctevelopment was 
accorrplished iji two stages. First, information structures 
vTere derived for each of the two instructional packages by 



the laultidiinensional scaling of similaritY judgments made 
by six "experts" to all possible pairs of the seventeen 
key concepts. The coordinates of the concepts located in 
these derived information structures were then sulmtted 
to two oorputer based algorithmic procedures for producing 
instructional sequences. One algorithm produced sequences 
in which concepts highly related by the experts were 
located oontiguoi^sly in the sequence (t>xe proximal sequence) . 
The second algorithm produced sequences in viuch oonoepts not 
highly related were located contiguously (the non-proximal 
sequence) . Exaitples of the output of the algorithms are 
presented in Annex B of the Phase I Report. 

Dependent Measures . Four dependent measures were used 
in this experiment, exairples of vdiich are pjresented in Annexes 
D and E of the AFHRL-TR-73-51 (I) . 

(1) Similarity or relatedness jvidgrrents (on a scale 
from 1 to 100) between all possible pairs of the key 
oonoepts. These judgments \flere conpared with those produced 
by the six "es^jerts" for each package. 

{21 Sets of verbal multiple choice questions used by 
the Air Force in the assessment of student performance on 
the instructional packages. 

(3) Pictorial/graphic versions of the sets of verbal 
multiple choice questions. These were created to provide 
a ireasure conpatible to the pictorial/graphic presentation 
fontat. 

(4) A concept cloze coitprehension measure in vdiich 
all occurrences of key concepts in the original Air Force 
text were replaced by letters used as a substitution code. 
The stibjects' task was to match the code letters with the 
appropriate concepts presented previously d'':ring Hie 
instructional stage. 

The standard cloze technique was not included in this 
experinent due to tine constraints on the e^^^erimental 
sessions and due to the high intercorrelation (.76) found 
between the concept cloze and the standard cloze in Pilot 
Stu:^ 1. 
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Individual Djf f^renoe Msasure > All subjects were given 
the Delta Reading Vocabulary Test at the tune of the e>?)eri- 
inent. This test serves as oixr prinary neasure of intellectual 
ability. 

Procedure . The experiment was run in eight, two-hour 
ejqperimental sessions with all twelve ceils of the design 
being represented in each session. For each session, two 
e5^>erimenters and bvo separate classrooins were \ased. Becaiase 
the length of the two instructional packages varied slightly, 
the subjects in the cells utilizing the Characteristics of 
IVIatter naterial were run in a separate roam from those 
subjects reoeivdLng the Oscilloscope Operation package. Upon 
arrival to the experiinent, subjects were assigned to one 
of the two rooms at random and then assigned to one of the 
six conditions existing under each level of the instructional 
package factor. Again, these latter assignments were done 
randomly within the constraints of the distributional require- 
ments (see the discussion of the es^jerimental subjects 
presented earlier) . 

Following assignments to conditions, the subjects were 
given a brief description of their task^ Each subject then 
received a stimulus booklet which contained the instructional 
information in either verbal or pictorial/graphic form, and 
in one of the three sequences (original, proximal, or non- 
proximal) depending on the experimental condition assigned. 
The experimenter then paoed the subjects through the 
booklets at a rate of 45 seconds per page (this presentation 
rate was selected on the basis of Pilot Stvidy 1) . If a 
subject finished stacking a page before time was called he 
was instructed to think over previo\as material, but not 
to look elsei^^iere in his booklet. 

FollcvTing the oartpletion of the stimulus booklet, eacii 
subject was given the foior dependent measvores in an order 
viiich would minimize the transfer of content-specific 
information from one measure to the next. This order was: 
similarity judgments, pictorial multiple choice, verbal 
multiple choice, and oonoept cloze. Instructions for each 
of the measures were priated on the tests booklets. Each 
subject was allowed to proceed through these measures at 
his own rate. 
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After corrpleting all of the response rneasures, each 
siabject ;vas given the Delta Reading Vocabulary Test. 
Carpletion of this test cxDncluded the experinental session. 

Results 

The presentation of results v/ill be s\±>divided into 
discussions of the follovdLng facets of the data analysis: 

(1) Scoring of the dependent and individual difference 
nveas\ires, 

(2) four way analyses of variance, 

(3) interoorrelations anong the dependent and individual 
difference measures, 

(4) two way analyses of variance using nale/fernale and 
type of instructional package as factors, and 

(5) two way analyses of variance using RCXDC/non-ROTC 
and type of instructional package as factors. 

Scoring of the Dependent and Individual Difference 
Msasiares. Three of the dependent measures , the pictorial 
multiple choice, the vea±)al multiple choice, and the 
concept cloze were scored as prcportian correct in order 
to form a basis for ooKparison across conditions vMch 
required a slightly different total nurtiber of responses. 
The Delta Reading Vocabulary test was also scored as 
proportion correct. 

The similarity (or relatedness) judgments for each 
sx4)ject were correlated with the average judgments for the 
"experts" on the appropriate instructional package. These 
individual correlations then served as measures of the 
correspondence of the subject and e:^)eri: views of the rela- 
tionships beto^een the key ccnoepts. Presumably, the more 
a subject learned from the instructional material, the 
greater would be his correlation with the average "es^rt." 
This measure will be referred to in the remainder of the 
report as S/IE Correlation . 
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The Four Way Analy^eg, ' of Varianoe , A four way, between- 
subjects, fixed effect analysis of variance was perfonned 
on each of the four dependent variables. The factors included 
in these analyses were: (a) package difficulty (Characteristics 
of Matter versios Oscilloscc^ Operation) , (b) mode of 
presentation (verbal versus pictorial) , (c) instructional 
sequence (original versus proximal veri. is non-proximal) , and 
(d) Delta Evading Vocabulary perfontanoe (high versus lew) • 
Levels of this last factor were formed by assigning the seven 
best performers on the Delta within a cell to the high gro\:p 
and the seven worst to the low groi:qp. The individual with 
inedian perfonnanoe on the Delta was excluded from the analysis. 
Inclusion of this Delta VocabuJ.ary factor in the analysis 
served to potentially reduce the anount of variance entered 
in the error term^ thus iitprovdng the pcwer of the analysis. 

Rather than treat each of the analyses separately, the 
results for each factor on each dependent measure are presented 
contigiaoiosly. The ireans and standard deviations for each cell 
on each dependent measure are presented in the Annex A of this 
report. 

The package difficulty factor led to significant results 
on all four dependent measurers: pictorial multiple choice, 
F(l,144 d.f.) = 110.4, p< .001; verbal multiple choice, 
F(l,144 d.f.) = 88.0, p<.001; concept cloze, F(l,144 d.f.) = 
102.5, p< .001; S/E correlations, F(l,144 d.f.) = 58.0, 
p < .001. The mean performanoe on the tstJO packages for 
each dependent measure are presented in Table 1. Inspection 
of the means reveals that for all the dependent measures, 
except the S/E pDrrelation, the CSiaracteristics of Matter 
package led to better perfonnanoe than did the Oscilloscope 
package. These findings are congruent with the results of 
our pilot work. The S/E correlation measure, hcwever, 
reverses the above findings, with the Oscilloscope package 
leading to significantly larger average correlations than 
the Characteristics of Matter package. 
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TABLE 1 



Four Way Analyses of Variance: Group Proportion Means for Package 

Difficulty Factor 
(proportion correct except where indicated) 





Package Difficulty 


Dependent Measures 


Characteristics 


Oscilloscope 




of Matter 


Operation 


Pictorial Multiple 
Choice^^ 


.778 


.548 


Verbal Multiple 
Choice* 


.800 


.589 


Concept Cloze^^ 


.845 


.549 


Subject /Expert 
Correlations* 


.130 

(average correlation) 


.266 

(average correlation) 



N = 84 Ss per condition 
* p< .001 



This last result is probably due to the greater adichness 
of relationships among the Characteristics of Matter oonoepts. 
These are basic physics concepts and many of the actual 
relationships v;ere not explicit in the instructional material. 
It is possible that the "experts" , having had more physics 
training, were basing their judgitents on relationships not 
directly available in the instructional material. Also, since 
nost of the subjects had some prior training in elementary 
physics, their judgments may have been biased by prior know^ 
ledge. On the other hand, the Oscilloscope package appeared 
to e:5q>licitly contain the majority of possible relationships 
between concepts, thus putting the subjects and "experts" on 
a coimoa ground. With this package a more valid assessment of 
subject-"e5^3ert" correspondence seemed to be achieved. 
Differences between packages on the S/E Correlation Ifeasure 
v?ere probably spuriously generated by differences in relational 
richness rather than package difficulty. 
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The irode of presentation factor was found to be irarginally 
significant with the pictorial imiltiple choice measure (F (1,144 
d.f.) s= 3.5, p <.075) and nonsignificant with the three other 
dependent ineasures. With pictorial imiltiple choice, the pictorial 
presentetion mode led to better performance than verbal presenta- 
tion (see Table 2) . 

TABLE 2 



Group Means for the Main Effect of Presentation Mode on All Four 

Dependent Measures 
(proportion correct except where indicated) 



Dependent Measures 


Mode of Presentation 


Pictorial/Graphic 


Verbal 


Pictorial Multiple Choice* 


.684 


.642 


Verbal Multiple Choice 


.705 


.686 


Concept Cloze 


.715 


.700 


S/E Correlations 
(Average Correlation) 


.211 


.188 



N = 84 Ss per cell 
* p <.075 



Inspection of the means in Table 2 indicates t!iat, althou^ 
not significant, the level of performance v/ith the Pictorial/ 
Graphic mode of presentation was higher on each of the remaining 
dependent measures. Since the Verbal Multiple C3ioioe, Concept 
Cloze, and S/E Correlation measures are in a verbal format, 
perhaps the failure to achieve statistical significance with 
these neasures is due to difficulties encountered by subjects 
in translating the Pictorial/Graphic information into a format 
ccnpatible with the verbally based dependent measures. This 
notion will be explored further in the Discussion section of 
this report. 

The instructional sequence factor was significant with the 
Verbal Multiple Choioe measure {F{2,144 d.f.) = 3.6, p < .05) 
and nonsignificant with the three remaining dependent variables. 
The groi:^:* means are presented in Table 3. As can be seen, exoept 
for the S/E Correlation measirre, the Proximal sequence generally 
results in slightly better performance than the other two sequenois. 
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A Scheffe^ post hoc analysis on the Verbal I^ultiple Choice ireasure 
revealed that the Proximal was significantly different from the 
Non-proximal at the .05 level and that other differences between 
means v?ere nonsignificant. 

TABLE 3 



Group Means for the Main Effect of Instructional Sequence on 

All Four Dependent Measures 
(proportion correct except where indicated) 





Sequence 


Dependent Measures 


Air Force 






Original 


Proximal 


Non-ProXimal 


Pictorial Multiple Choice 


.663 


.670 


.656 


Verbal Multiple Choice* 


.698 


.730 


.658 


Concept Cloze 


.685 


.712 


.687 


S/E Correlations 


.214 


.196 


.188 


(Average Correlation) 









N - 56 Ss per cell 
* p<.05 



Potential reasons for the lack of statistical significance 
vdth the Concept Cloze and S/E Correlation dependent measures 
on the instructional sequence factor can be cited. First, the 
previoxasly discussed difficulties with the S/E Correlations and 
the Characteristics of Matter package vwDuld serve to reduoe 
sensitivity to instructional sequence. Second, inspection of 
the Cdnoept Cloze test showed than in some conditions the 
sensitivity of this ineasure was being attenuated by "ceiling 
effects"; a nunber of subjects were scoring 100% on this test 
with the Characteristics of Matter Instanactional iraterial. The 
failure of the Pictorial Multiple Choice ineasure to reflect 
significance, however, does not seem to be attributable to 
difficulties with the ineasure. Qbvioxasly, fiarther enpirical 
exploration is necessary. 
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Significant differences due to the Delta Eeading Vocabulary 
performance factor were found for each of the four dependent 
measures (Pictorial Multiple Choice, F(l,144 d.f.) = 38.3, 
p< .001; Verbal Multiple Choice, F(l,144 d.f.) == 31.0, p< .001; 
Concept Cloze, F(l,144 d.f.) ^ 52.8, p < .001; S/E Correlations, 
P(l,144 d.f.) = 15.0, p< .001). Inspection of the perfomance 
means revealed that with all dependent variables the high 
scoring Delta siibjects perforned better on the dependerit measures 
than the low scoring Delta subjects. 

The only interac±ion effect reaching significance in the 
four way analyses was package difficulty by Delta perfonnanoe. 
This effect was significant with the S/E Correlation measure 
(F(l,144 d.f.) == 8.8, p 4^.005) and nonsignificant with the 
three other dependent variables. The means representing the 
interac±ions are presented in Table 4. As can be seen the 
significant difference between, high and Icm Delta on the 
S/E Correlation measures is primarily due to performance on 
the Oscilloscope Operation package. This finding adds further 
evidence to the previously discussed notion that the S/E 
Correlations are only valid with the Oscilloscope material. 

TABLE 4 



Four Way Analyses of Variance: Group Means for the Interaction 
Between Delta Performance and Package Difficulty 





Pictorial Multiple Choice 
(Proportion Correct) 




Characteristics of 
Matter 


Oscilloscope 
Operation 


^High Delta Performers 


.858 


.604 


Low Delta Performers 


.698 


.493 


N = 42 Ss per Cell 




Verbal Multiple Choice 
(Proportion Correct) 




Characteristics of 
Matter 


Oscilloscope 
Operation 


High Delta Performers 


.881 


.633 


Low Delta Performers 


.718 


.549 



N = 42 Ss per Cell 
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TABLE 4 (Cont.) 





' Concept Cloze 

(Proportion Correct) 




Characteristics of 
Matter 


' Oscilloscope 
Operation 


High Delta Performers 


.934 


.671 


Low Delta Performers 


.756 


.418 


N 42 Ss per Cell 




S/E Correlation 
(Average Correlation)* 




Characteristics of' 
Matter 


" Oscilloscope 
Operation 


High Delta Performers 


.139 


.328 


Low Delta Performers 


.123 


.206 



N 42 Ss per Cell 
* P<.0a5 



Intercx^rrelations Among the Dependent and Individxial 
Difference Measxares . The intercorrelations aitong performanoes 
on the four dependent ineasures and the Delta Ifeading Vocab\xLary 
test were calaalated for each of the ta^lve cells of the three 
factor design. Ctrude averages of these correlations were then 
calculated to produce intercorrelation matrices for each level 
of each factor. 

The average intercorrelation iratrioes for the two instruc- 
tional packages are presented in Table 5^ Inspection of these 
matrices reveals a number of things. First, the oorrelaticns 
of the S/E Correlation measure with the other dependent variables 
are negligible for the Characteristics of Matter package but 
itoderate for the Oscilloscope material, again supporting the 
raticn of differential sensitivity of this msasxjre across packages. 
Second, the intercorrelations among the other three dependent 
measures are higher with the Characteristics of Matter package 
than with the Oscilloscope material. The greater difficulty of 
the Oscilloscope package may have led to a larger degree of random 
responding, thus attenuating the correlations. FincLLly, with the 
exception of the S/E Co3rrelatian measure, the correlations between 
the Delta and the other three dependent vari^les range from . 35 
to .60, with no large, consistent differences between packages. 



18 



TABLE 5 



Average Intercorrelation Matrices for the Two Levels of the 

Package Difficulty Factor 



'"'ARACTERISTICS OF >IATTER 





Pictorial 


Verbal 


Concept 


S/E Corr. 


Delta 


Pictorial Multiple 

Choice 


1.0 










Verbal Multiple 
Choice 


.84** 


1.0 








Concept Cloze 


.74** 


.69** 


1.0 


J 
i 


Subject/Expert 
Correlations 


- .01 


.06 


.06 


1.0 i 


Delta Reading 
Vocabulary 


.55** 


.60** 


.49** 


.07 i 1.0 



N = 84 Ss 
** p<.01 



OSCILLOSCOPE OPERATION 





Pictorial 


Verbal \ 


Concept 


S/E Corr. 


Delta 


Pictorial Multiple 
Choice 


1.0 


i 

! 








Verbal Multiple 
Choice 


.35** 


1.0 








Concept Cloze 


.60** 


.36** 


1.0 






Subject/Expert 
Correlations 


.30** 


.35** 


.65** 


1.0 




Delta Reading 
Vocabulary 


.35** 


.36** 


.54** 


.35** 


1.0 



N = 84 Ss 
** p^.Ol 



Uie average intercorrelation matrices for the levels of the 
mode of presentation facrtor are presented in Table 6. The main 
discrepancy of interest beto^^een these •toro matrioes ijnvolves correla- 
tions with the Delta Reading Vocabulary. (The matrioes also shew 
differences in the S/E Correlation measiore, but the difficulties 
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vdth this msasure have already been discussed.) The correlations 
beta'^een the dependent variables and the Delta are substantially 
larger with verbal presentation than \vith pictorial. This 
difference itakes, sense in that a ireasure of verbal ability, such 
as the Delta, should be more predictive in a verbal test than in 
a pictorial. Presumably, if we had included a measure of 
spatial/pictorial ability, the reverse finding woxild have occurred. 

TABLE 6 

Average Intercorrelation Matrices for the Two Levels of the Mode 

of Presentation -Factor 



PICTORIAL PRESENTATION 





Pictorial 


Verbal 


Concept 


S/E Corr. 


Delta 


pictorial Multiple Choice 


1.0 










Verbal Multiple Choice 


.61** 


1.0 








Concept Cloze 


.73** 


.60** 


1.0 






Subject/Expert Corr., 


.26** 


.32** 


.49** 


1.0 




Delta Reading Vocabulary 


.38** 


.43** 


.46** 


.25* 


1.0 ' 



N = 84 Ss 
* p < .05 
** p < .01 



VERBAL PRESENTATION 





' Pictorial 


Verbal 


Concept 


S/E Corr. 


Delta, 


Pictorial Multiple Choice 


1.0 










Verbal Multiple Choice 


.58** 


1.0 








Concept Clo ze 


.62** 


.46** 


1.0 






Subject/Expert Corr. 


.02 


.09 


.22* 


1.0 




Delta Reading Vocabulary 


.51** 


.53** 


.57** 


.17* 


1.0 



N = 84 Ss 
* p < .05 
** p < .01 



The average interconrelation niatrices for the instructional 
sequence factor are presented in Table 7. These matrices appear to 
be very similar to one another with no oaipelling discrepancies. 
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TABLE 7 



Average Intercorrelatlon Matrices for the Tliree Levels of the 
Informational Sequence Factor 



ORIGINAL AIR FORCE SEQUENCE 





Pictorial 


Verbal 


Concept 


S/E Corr. 


Delta 


Pictorial Multiple Choice 


1.0 










J/erbal Multiple Choice 


.65** 


1.0 








Concept Cloze 


.71** 


.60** 


1.0 






Subject/Expert Correlations 


.15 


.22 


.42** 


1.0 




Delta Reading Vocabulary 


.47** 


.52** 


.60** 


.23* 


1.0 



N 56 Ss 
* p .05 
** p < .01 



PROXIMAL SEQUENCE 





Pictorial 


Verbal 


Concept 


S/E Corr. 


Delta 


Pictorial Multiple Choice 


1.0 










Verbal Multiple Choice 


.62** 


1.0 








Concept Cloze 


.65** 


.51** 


1.0 






Subject/Expert Correlations 


.02 


.05 


.34** 


1.0 




Delta Reading Vocabulary 


f .55**' 


.46** 


.54** 


.18 


1.0 : 



N = 56 Ss 
* p < .05 
** p < .01 



NGN PROXIMAL SEQUENCE 





Pictorial 


Verbal 


Concept 


S/E Corr. 


Delta 


Pictorial Multiple Choice 


1.0 










Verbal Multiple Choice 


.51** 


1.0 








Concept Cloze 


.65** 


.48** 


1.0 






Subject/Expert Correlations 


.25* 


.36** 


.30* 


1.0 




Delta Reading Vocabulary 


.34** 


.47** 


.41** 


.22 


1.0 



N 56 Ss 
* P < .05 
** p ^ .01 
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Two ' Way Malysea ' of • yariano^ ' ( Malea/Femaleg and Package 
Difficulty} , A two-way, between subjects, fi:xed effects 
analysis of variance was calculated for each of the four 
dependent iteasures with males (122 siibjects) / females (58 
subjects) and Package Difficulty as the factorFi. Rather than 
treat each of the analyses separately, the results for each 
factor on each dependent measxure will be presented oontiguoiasly. 

As with the four-way analyses of variance the main 
effect of package difficulty was significant on all four 
dependent measxires at the .001 level. The Characteristics 
of Matter package lead to better performance than the 
Oscilloscope instructional material. 

The Male/Female main effect was significant v/ith all four 
dependent measures (Pictorial, Multiple Choice, F (1,176 d.f.) - 
3.8, p 41 .075; Verbal Multiple Choice, F(l,176 d.f.) = 5.3, 
p 4. .025; Conoept Cloze, F(1,17G d.f.) = 7.5, p < .01; S/E 
Correlations, F(l,176 d.f.) = 22^4, p <.001). Inspection of 
the means indicates that in all cases tiie males performed at 
a. higher level than the females. 

The interaction betaween Male/Female and Package Difficulty 
was significant on all of the dependent measures except the 
Verbal I^tiple cEoIce test (Pictorial Multiple Choice, 
F(l,175 d.f.) = 4.2, p < .05; Concept Cloze F(l,176 d.f.) = 
12.3, p< .001; S/E Correlations, F(l,176 d.f.) = 12.8, p< .001. 
The means involved in these significant interactions are presented 
in Table 8. As can be seen from the table, the differences 
between males and f etales is in performance on the Oscilloscope 
material. Again, it should be noted that the S/E Correlation 
measure is sensitive to differences with the Oscilloscope 
instructional material. 
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TABLE 8 



Two Fay Analyses of Variance; Group Means for the Significant 
Interaction Between the Male/Temale Factor and Package Difficulty 



PICTORIAL MULTIPLE CHOICE** 





Characteristics of Matter 


Oscilloscope Operation 


Males 


.793 (N = 57 Ss) 


.568 (N = 65 Ss) 


Females 


[ .752 CN = 33 Ss) 


.497 (N = 25 Ss) 



** p <C.05 



VERBAL MULTIPLE CHOICE 





Characteristics of Matter 


Oscilloscope Operation 


Males 


.822 (N = 57 Ss) 


.599 (N = 65 Ss) 


Females 


.762 (N = 33 Ss) 


.565 (N = 25 Ss) 






CONCEPT CLOZE* 






[characteristics of Matter 


Oscilloscope Operation 


Males 


.853 (N = 57 Ss) 


.592 (N = 65 Ss) 


Females 


.832 CN = 33 Ss) 


.436 CN = 25 Ss) 



* p < .001 



SUBJECT/EXPERT CORRELATIONS* 





CharacteriE 


itics 


of 


Matter 


■ Oscilloscope 


Operation 


Males 


.137 


CN = 


57 


Ss) 


.299 


CN = 


65 Ss) 


Females 


I .118 


(N = 


3i 


Ss) 


.180 


(N = 


25 Ss) 



* p^.OOl 



. Two Way Analyses of Varianoe ( JMale I^^/Male Non -HDTC) . 
A taTo-way, between sxabject, fixed effects analysis of varianoe 
was performed on each of the four dependent neasiices with Male 
Km: (50 subjects)/ Male Non-RDOX: (72 subjects) and Package 
Difficulty as the factors. Again, the resiiLts for each factor 
on each dependent measure will be presented contiguously. 
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.As- expected the effect of package difficulty was again 
significaat at the .001 level' on all four dependent measures. 

The effect of the Male ROTC/Male Non-BDTC factor was 
significant m,th the Pictorial Miltiple C3ioioe (F (1,118 d.f.) = 
8.8, p < .005) and the Verbal Multiple Choice (F(l,118 d.f.) = 
5.3, p .025) and was nonsignificant with the Concept Cloze 
and the S/E Correlations, inspection of the significant ireans 
(Table 9) reveals that the EDTC males perfonned at a higher 
level than the Ifcai-POTC males. The interaction effects wab 
nonsignificant with all dependent measures. 

TABLE 9 



Group Means for Male ROTC/Male Non-ROTC Factor on Each Dependent 

Measures 

(Proportion Correct Except Where Indicated) 



Dependent Measure 


MALE ROTC 
N = 50 


MALE NON-ROTC 
N = 72 


Pictorial Multiple Choice** 


..699 


.655 , 


Verbal Multiple Choice* 


.719 


.703 


Concept Cloze 


.735 


.698 


S/E Correlations 
(average correlation) 


.236 


.211 



* p<.025 
** p< .005 



Very little weight should be given to the f indijig of 
significant differences between RDOX: and Non-POTC males for 
t^rJO reasons. First, the distribution of individiaals frcm 
these two categories over e35)e2±mental treatments was rela- 
tively uneven {due to constraints of the exferimental 
situation) . Thvis, treatnent effects would naturally confound 
these differences. Second, the motivational level of the 
two categories of subjects may have varied due to the 
military nature of the e3^)erirnent. 
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Discaassion 



The discussion of results will be divided into foiar 
si±>~secrtions dealing \^d.tI^ the follc^^ing topics: the iirpact 
of instructional sequencing, the effect of presentation 
mode, the role of individual differences, and the effectiveness, 
of the dependent measures. 

Bef02:e proceeding with the discussion it is irrportant 
to point out that a number of other analyses including 
analyses of oovariance, were performed on the data. Because 
these analyses provided information that was redundant to 
the reported results, they are not presented in this document. 

The Iirpact of Instructional Sequencing . The primary 
pxorpose of this research was to study xnstructional sequencing 
in a context vMch resolved some of the difficulties cited 
in the literature review and synthesis (AEHRL-TR-73-51 (I) . 
In particular, the aim was to study the effects of objectively 
derived sequences (multidimensional scaling methods) and 
their interaction with individual difference variables and 
other meaningful instructional manipulations. The result of 
the fo\ar way analyses of variance indicated that a main effect 
oarposed of three different instructional sequences was signi- 
ficant with the Verbal Multiple Choice IVieasure and that all other 
interactions involving this factor were nonsignificant. 

In addition, inspection of the means (Table 3) associated 
with the main effects on the other three dependent measures 
indicated that, iii general, as with the Verbal Multiple Choice 
test, the proximal sequences (generated by the multidiirensional 
scaling methods) led to better performance than the other two 
sequences. 

As stated in the presentation of results, the failure of 
the main effect of three instructional sequences to reach 
significance on the other dependent variables is partially 
explainable in terms of the present shortcomings of these 
other measures. The Concept Cloze led to skewed results 
(ceiling effects) with the Characteristics of Matter package, 
thios reducing its sensitivity to diffe3:enoes in performance. 
Further, the S/E Correlation variable has been sham to be 
ins^sitive to performance differences with the Characteristics 
of Matter material. This finding will be discussed more 
thoroughly in the sub-section entitled: The Effectiveness 
of the Dependent Measures. 
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Mthqugh one cannot be certain, it seems that the lade 
of correspondence among dependent jiBasvires \dth the instructional 
sequencing variable is not due to a spurious effect sha>7n with 
the Verbal Multiple Choice variable, but rather to the relative 
insensitivity of the iremaining measiures Crernertiber that the 
direction of differences between ineans v/as similar for all 
measures exoept the S/E Correlation) . It should be noted that 
the Verbal Multiple Choice Measure contains the sets of 
questions originally used by the Air Force in testing acquisi- 
tion of the instructional materials. Presumably this measure 
has been examined much more thoroughly for reliability and 
validity than the three other prototype measures especially 
constructed for this experiment. 

Post hoc analysis of the effect of instructional sequencing 
with the Verbal Multiple Choice Variable indicated that thie 
Proximal sequence derived from the multidimensional scaling 
method led to significantly better performance than the Non- 
Proximal Sequence. Inspection of the means revealed that the 
Original Air Force Sequence produced a level of student 
performance vdtiich fell between the Proximal and Nbn-Proximal 
sequences. The lack of larger differences in mean perfonmnce 
between the proximal and original Air Force Sequence conditions 
is not surprising. First, the original Air Force sequences have 
undergone revisions based on prior instructional experiences 
(revisions of some of this material probably occurred even 
prior to incorporation of these packages by the Air Force) . 
These seqi:ences are now undoubtedly close to being optiiral for 
performance effective instruction. In fact, the actual 
sequence differences beta^een the proximal and the original 
were quite sx4)tle, therefore limiting potential performance 
differences. However, with newer instructional material (that 
is, material that has not been heavily revised based on 
instructional e^qperience) it would be ejected that the 
proximal sequence generated by the objective raoltidimensional 
scaling approach would lead to substantially better performance 
than a sequence generated by subjective means. 

A second reason for the small performance differenti.al 
in favor of the proximal sequence is the size of the instruc- 
tional unit treated in this e:q)eriment. Due to pragmatic 
constraints, only about 30-40 minutes of instructional material 
could be presented. It is probably true that the iitpact of 
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sequencing is inuch stronger over larger lanits of instruction. 
The itiultidijTBnsional scaljjig method can be applied to all 
levels of instruction f ran the overall curriculum to a 
particular course to a specific lesson. If there are performance 
differences in favor of this method with small lonits of 
instruction then these differences should be airplif ied with 
larger units. 

Even if the objective laul'tidimensional scaling raethod 
proves only equal in educational effectiveness to the present 
subjective techniques xased by cixrriculum designers, the method 
iTHv \\7ell offer advantages in efficiency, replicability, and 
control. With subjective techniques, the resxilting sequence 
depends heavily on the skill of the individiial designer, vMle 
the objective sequencing method p^rniits the views of a Isorge 
number of "experts" to be ocsTibined at a very small cost. 
Because the latter method is <S?jective and allows for the 
pooling of a number of opinions, costly c±ecking and re-checking 
of the effectiveness of instruction may be dramatically reduced. 

The lack of significant interaction beto^?een iiistructional 
sequence and \^iables irepresenting both individual differences 
and other instructional manipulations (for exaitple, mode of presenta- 
tion and package difficulty) may be most likely due to one of 
t^jo reasons. First, the dependent measures eirployed may not 
have been sufficiently sensitive to detect such effects. 
Second, with the sets of instructional materials errployed in 
the present stxacty, it nay be true that such interaction effects 
do not exist at a detectable level. In either case, fxjQrtJier 
studies should be undertaken using a wider range of instructional 
material (especially material that has a wide irange of a priori 
familiarity associated with it) and a greater variety of 
dependent measiores. 

The Effect of Presentation Mpde . The results of the foiar 
way analyses of variance indicated that with the Pictorial 
Multiple Choice test Pictorial/Graphic presentation leads to 
significantly better performance than Verbal presentation 
(marginally at the .075 level) . Although this main effect did 
not reach significance with the other three dependent measvires, 
inspections of the means in Table 2 reveals general agreement 
among the measures with regard to the direction of mean differ- 
ences. 
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Perhaps the f ailiore of the other iteasiuBs to reach signifi- 
cance "was due to the difficulties on the part of the Pictorial/ 
Graphic presentation g2X)i:ips in translating the learned material 
into the verbal format required by the Verbal Multiple Choice, 
Concept Cloze, and S/E Correlation measures. In any case, the 
very tentative conclusion that the Pictorial/Graphic mode of 
presentation leads to better performance than the Verbal node 
requires further exploration and delimitation in the cQntex-t of 
a variety of instructional materials. 

The Eole of Individual Differences . Besides their potential 
interaction with instanictional sequence, individual, differences 
are of general inportance in the context of instnaction. In this 
study three sources of individual differences v?ere examined: 
performance on the Delta Reading Vocabulary test, sex, and EDTC 
affiliation. As mentioned in the presentation of results, the 
effects of EOTC affiliation v;ei:e potentially confounded by 
differential motivational properties of the task for vario\as 
classes of individuals and distributional imbalances across the 
experimental treatment groijps. Because of these difficulties 
there will be no further discussion of this effect. 

The results of the f oiar-way analyses of variance indicated 
that high scorers on the Delta measure performed substantially 
better on all dependent variables thar>, did low scorers. The 
one significant interaction involving Delta was with package 
difficulty on the S/E Correlation measure. As stated previously, 
this effect is probably dxae to the differential sensitivity of 
this dependent measure wi.th the two levels of instructional 
material rather than a true interaction between Delta and 
package difficulty. 

The interoorrelations between Delta and the dependent 
measures generally fell between .35 and . 60. These c6rrelations 
were sanev*iat higher for the verbal rr^^de of presentation and 
scBn[evdiat lower for pictorial presentation. Since the Delta 
is a p^)er and pencil test requiring only 10 minutes for admin-- 
istration, it would appear to have sxibstantial valine as a device 
for selecting students and as a device for determining the 
sequence and pace or rate of instruction. This would be 
especially true for verbal materials. 
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The analysis of male/female differences in performancse 
genercilly indicated a large difference in favor of the iriales 
on the Oscilloscope Operation package and a substantially 
smaller difference also in favor of the males, on the Charac- 
teristics of Matter material. The Oscilloscope Operation 
package led to significantly wrse performance th*an the 
Characteristics of jyiatter on all dependent meas\3res. The 
difficulty of this package and the relatively unfamiliar 
scientific concepts involved may have caused the greater 
differential between males and females due to a priori 
differences in scientific interest and ability. Such differ- 
ences should certainly be taken into account in the design 
of future instructional programs. 

Effectiveness of the Dependent Measures . The Verbal and 
Pictorial Multiple Choice Measures appeared to give consistent 
results in a range maffected by "ceiling" and "floor" effects. 
However, as witii the two other measures, there may be sane 
question as to their effectiveness with modes o£ presentation 
that are not conpatible with the format of the measures. This 
notion has been discussed earlier in the section entitled: 
The Effect of Presentation Mode (see page 27) . 

Rirther oomnents iregarding the effectiveness of the Concept 
Cloze and S/E Correlation measures are necessary. The Concept 
Cloze test appears to have much promise as an alternative to 
the Standard Cloze technique. As with the Standard Cloze it is 
objectively constructed and scored,, but unlike the standard 
approach, it only tests knowledge of important concepts, not 
knowledge associated with a combination of irrportant concepts 
and trivial function words. In this sttriy, the only difficulty 
with the Concept Cloze stenmed from the "ceiling effects" 
found with the verbal presentation of the Characteristics of 
Matter material. This sitiiation could easily be alleviated 
by asking the subjects to fill in the appropriate key concepts 
in a segment of parallel text not xised during the instiniction 
biit whicih contains the same material. This would increase 
the dif f icul1^ of the task over that vMch ex?.sts with the text 
material prevj.ously used during instruction. 
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As discussed in the presentation of the results, the 
S/E Correlation ineasxire only appears to be sensitive and 
consistent with the Oscilloscope Operation package. Within 
this package this measure has substantial intercorrelations 
vdth the other dependent measures and, on the basis of the 
magnitude of tJie F ratios, it may in fact be a more sensitive 
indicant of jperformance. Although the S/E Correlation m^asiire 
has promise, further exploration of its limitations, especially 
with varieties of instructional material, is necessary. 

The intercorrelations between the dependent measures, in 
particular the Verbal Multiple Choice, the Pictorial Multiple 
Choice, and the Concept Cloze, generally fell between .60 
and .70. This level of relationship is sufficient to provide 
scire evidence of concurrent validity, but not high enough to 
recomnnend the eliinination of sons of the measures on the 
basis of redundancy. A set of measures of this type may be 
very useful for assessment and prescription in an instructional 
setting. 

Sxanmary and Concl'usions 

The pxirpose of this investigation was to explore the 
effects of instructional sequence on perfoinriance in an 
e^q^erimental context which presumably corrected some c X the 
problems plaguing prior research in this areri. In particular, 
the e^q^eriment was designed to accoitplish the following: 

(1) To assess the effectiveness of instructional 
sequences generated by a multidimensional scaling approach; 
a new approach v^ich can objectively synthesize the views of 
a number of curriculim "e>^rts", 

(2) To utilize newly developed dependent, measures in an 
attenpt to provide more sensitive indicants of performance 
differences, 

(3) To fxHTther our knowledge of the interaction of 
individual difference variables and sequencing, 

(4) To assess the interaction of instructional sequence 
effects with t^/o other educationally irrportant variables: mode 
of presentation and instructional difficulty. 



Although not definitive, the resiilts of tMs experiirent 
do provide evidence for the utility of the INSCAL raultidimen-- 
sional scaling approach in describing inf onnatdLon structxares 
and generating instructional sequences. Generally, the irean 
performances with the proximal sequence (a presumably optimal 
sequence generated by the multidimensional scaling approach) 
were higher than perfoirmances with both the original Air Force 
sequence, and the non-proximal sequence (a second sequence 
generated by multidimensional scaling using a different set 
of generati.on rules) . It would be expected that performance 
advantages associated with the proximal sequence would be 
further enhanced in ooiTparisons with newer Air Force instruc- 
tional material (material in which sequence had not been 
heavily modified on the basis of prior instructional esqperience) 
and with larger instructional lanits. In any case, the INSCAL 
multidimensional scaling method for developing instructional 
sequences offers some unique advantages over the traditional 
siiDjective approaches: 

(1) The INSCAL approach can combine the views of a 
large number of "experts" to form a single sequence. Besides 
providing consensual validation, this feature has the psycho- 
logical advantage of involving a number of individuals in 
curriculum decision making, 

(2) The INSCAL approach can be applied to instruction 
at all levels; the overall curriculum, an individual course, 
and/or a specific lesson. 

(3) Since the INSCAL is sensitive to differences 
between individuals oontributing individuals, it is possible 
to objectively select a particular instiructional sequence 
for a specific si±>-gro\5). of instructors or students. 

(4) The INSCAL has the capability of providing an 
index of instructional oonplexity. Such an index would 
provide a strong basis for student selection and pacing. 

Overall, the INSCAL multidimensional scaling approach offers 
advantages sufficient to warrant further research and developrvent. 
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In addition to the effects of sequence, the present experi- 
ment sha(tfed consistent but small, perfoimance differences due 
to mode of presentation. The Pictorial/Graphic irod^ led to 
better perforrranoe than did the Verbal mode. Since the pictorial/ 
graphic format for the instructional material used has imdergone 
far less research and development than the verbal format, this 
result is quite encouraging. Further deveiopnent of the pictorial/ 
graphic foamat should be undertaken and its' interaction with 
individual difference variables (for exanple, mode preference) 
shoxild be assessed. . 

With regard to individual differences in this study, the 
Delta Reading Vocabulary test proved to be substantially related 
to performance with correlations generally ranging frrai .35 to 
.60 depending on the experimental oondition. Since this measure 
requires veiy little time to administer and score it voiald appear 
to be a \aseful adjunct to test batteries xased for student selection 
and prescription. 

The performance diffea:nenoes between males and females, in this 
e3<periitent were sx±>stantial (the males perfonning at a higher 
level than the females) . Clearly, sex should be taken into account 
in the design of individualized instruction, especially instruction 
involving scientific and/or technical content. 

The Concept Cloze test and the S/E Correlations weare t^ new 
dependent measures designed and constructed for xased in this 
experiment. Although both of these measures contained some flaws, 
they appear to have sufficient potential to warrant their further 
investigation as indicants of successful acquisition of instructional 
material. 
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ANMEX A 

MEANS MD STANDARD DEVIATIONS FOR EACH EXPERIMENTAL 
OOQDITICN CN ALL DEPENDENT MEASURES 
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performance Means and Standard on All Dependent Measures 
for the Original Air Force Seq^uence Condition 



(Proportion Correct Except for S/E Measure which 
Consists of Average Correlations) 



MODE OF 


PACKAGE 




DELTA PERFORMANCE 






PRESEMTATION 


DIFFICULTY 




















HIGH 


LOW 






Measure 


Mean 


S.D. 


Measure 


Mean 


S.D. 






PMC 


.590 


.096 


PMC 


.446 


.121 








VMC 


.623 


.108 


VMC 


.564 


.124 








CC 


.651 


.225 


CC 


.373 


.167 






S/E 


.350 


.172 


S/E 


.211 


.149 








D 


.768 


.073 


D 


.544 


.055 




VERBAL 




















PMC 


.907 


.089 


PMC 


.653 


.121 








VMC 


.931 


.077 


VMC 


.646 


.183 






L>HAKAC< JLJbKlo JL ILc 


CC 


.993 


.017 


CC 


.701 


,253 






OF MATTER 


S/E 


.111 


.027 


S/E 


.130 


.082 








D 


.876 


.048 


D 


.603 


,079 








PMC 


.630 


,128 


PMC 


.538 


,126 








VMC 


.643 


.100 


VMC 


.555 


.128 






OSCILLOSCOPE 


CC 


.647 


.244 


CC 


.410 


,192 








S/E 


.240 


.123 


S/E 


.304 


,143 








D 


.798 


.032 


D 


.658 


.095 




PICTORIAL/ 


















GRAPHIC 




PMC 


.810 


.126 


PMC 


.734 


,195 








VMC 


.838 


.091 


VMC 


.783 


,130 






CHARACTERISTICS 


CC 


.916 


.171 


CC 


.787 


,200 






OF MATTER 


S/E 


.154 


.075 


S/E 


.110 


,077 








D 


.853 


.046 


D 


.557 


,091 





N = 7 Ss Per Cell 

KEY 

PMC = Pictorial Multiple Choice 

VMC = Verbal Multiple Choice 

CC = Concept Cloze 

S/E - S/E Correlations 

D - Delta Reading Vocabulary 
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P.erfQrmp.nce W.eans and Standard Deviations on- All Dependent 
Measures for the Proxljnal Seq^uence Condition 

CProportion Correct Except for S/E Measure which 
Consists of Average Correlations) 



MODE OF 


PACKAGE 




DELTA PERFORMANCE 






PRESENTATION 


DIFFICULTY 


















HIGH 


LOW 






Measure 


Mean 


S.D. 


Measure 


Mean 


S.D. 






PMC 


.564 


.110 


PMC 


.487 


.123 






VMC 


.638 


.100 


VMC 


.638 


.216 




OSCILLOSCOPE 


CC 


.731 


.120 


CC 


• Jo/ 


1 AO 






S/E 


.316 


.112 


S/E 


• 181 


.132 


VERBAL 




D 


.808 


.048 


D 


.503 


.119 






PMC 


.887 


.092 


PMC 


.663 


.166 






VMC 


.933 


.060 


VMC 


.747 


.107 




CHARACTERISTICS 


CC 


.967 


.039 


CC 


.713 


.240 




OF MATTER 


S/E 


.120 


.035 


S/E 


.153 


.048 






D 


.818 


.035 


D 


.606 


.131 






PMC 


.643 


.139 


PMC 


.460 


.131 






VMC 


.628 


.176 


VMC 


.554 


.147 




OSCILLOSCOPE 


CC 


.691 


.193 


CC 


.477 


.181 






S/E 


.324 


.176 


S/E 


.197 


^121 






D 


.847 


.072 


D 


.601 


.088 


PICTORIAL/ 
GRAPHIC 


















PMC 


.094 


.069 


PMC 


.759 


.146 




VMC 


.937 


.056 


VMC 


.763 


.213 




CHARACTERISTICS 


CC 


.960 


.079 


CC 


.771 


.196 




OF MATTER 


S/E 


.158 


.047 


S/E 


.113 


.058 






D 


.824 


.062 


D 


.598 


.058 



N = 7 Ss Per Cell 

PMC 

VMC 

CC 

S/E 

D 



KEY 

Pictorial Multiple Choice 

Verbal Multiple Choice 

Concept Cloze 

S/E Correlations 

Delta Reading Vocabulary 



ERIC 
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Perforjoance Means and Standard Deviations on All Dependent 
Measures for ttie Nont-Proximal Sequence Condition 

(Proportion Correction Except for S/E Measure 
which Consists of Average Correlations) 



MODE OF 


PACKAGE 




DELTA PERFORMANCE 




PRESENTATION 


DIFFICULTY 


















HIGH 


LOW 








1 IC CtLI 


S.D 


1 ICOLS UX C 


"Mo a x\ 

1 IC Ct LI 


S D 






PMC 


597 


.139 


PMC 


• 488 


157 






VMC 


• 623 


.087 


VMC 


.478 


• 087 




OSCILLOSCOPE 


cc 


• \J *-r JL 


079 


CC 


313 


• 168 












S/E 


.104 


.064 






U 






u 




092 


VERBAL 




















PMC 


.747 


.139 


PMC 


.677 


.106 






VMC 


T r- 0 

.758 


.110 


VMC 


.648 


.161 




CHARACTERISTICS 


CC 


.813 


.252 


CC 


.817 


.090 




OF MATTER 


S/E 


.143 


.104 


S/E 


.123 


.075 






D 


.766 


.036 


D 


.510 


.155 






PMC 


.597 


.138 


PMC 


.538 


.098 






VMC 


.641 


.154 


VMC 


.504 


.054 




OSCILLOSCOPE 


CC 


.666 


.085 


CC 


.546 


.243 






S/E 


.333 


.092 


S/E 


.237 


.162 






D 


.848 


.036 


D 


.648 


.097 


PICTORIAL/ 
















GRAPHIC 




PMC 


.900 


.076 


PMC 


.701 


.208 






VMC 


.888 


.065 


VMC 


.717 


.222 




CHARACTERISTICS 


CC 


.954 


.055 


CC 


.747 


.255 




OF MATTER 


S/E 


.147 


.086 


S/E 


.107 


.091 






D 


.796 


.075 


D 


.513 


.103 



N = 7 Ss Per Cell 



KEY 

PMC = Pictorial Multiple Choice 

VMC = Verbal Multiple Choice 

CC = Concept Cloze 

S/E = S/E Correlations 

D = Delta Reading Vocabulary 
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